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ABSTRACT 

This paper describes a faculty-designed program 
evaluation procedure established at Appalachian State University in 
North Carolina, focusing on the Communication Department. The 
assessment potential of this process is examined and, following an 
explication of the process itself and the Program Evaluation 
Committee (PEC) Report, the five elements of the final report are 
explored: (1) centrality to the university's mission; (2) department 
overview; (3) faculty workload/reassigned time and scholarly 
activity; (4) program demand; and (5) program duplication. The paper 
outlines the eight-member PEC structure and functions, noting that 
the committee was not established as an assessment agency per se, and 
that its direct mission is summative rather than formative. The paper 
illustrates how this or similar program evaluation processes have the 
potential for assessing departmental effectiveness for departments 
wishing to take advantage of the opportunity. Academic program review 
evaluation criteria are appended. Contains 15 references. (CR) 
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"Evaluating Effectiveness Through Program Assessment" 



Since the early to mid 1980's, universities across the nation have responded to an 
increasing concern for assessment of academic programs, curricula, or institutional effectiveness 
(Higgerson, 1993, p. 1). While much of this concern has centered on student learning, other 
factors have included the nature of the undergraduate curriculum in general as well as increased 
demands by legislatures and the public for accountability (Hay, "Alternative Assessment," 1994. 
p. 1 and Higgerson, 1993, pp. 1,2). In 1987, the Southern Association of Colleges and Schools 
adopted a mandate for assessment. The North Carolina legislature passed an accountability 
mandate intended "to publicly demonstrate the effectiveness of higher education to the 
legislature and the general public (Appalachian State University, 1990)" in its 1989-1990 
session. Yet broader assessment efforts were called for by the Denver Conference Program 
Assessment Group: 

While assessment activities across the country, . . .have focused upon 
instruction and curriculum concerns, a total assessment effort needs to expand 
beyond these parameters and consider all of the department's endeavors. Once a 
department has a clear sense of its intentions, it must evaluate the extend to which 
its research, creative activity and service help to realize its agreed upon goals and 
objectives (1992, p. 11). 

Amid all of this concern for student outcomes, institutional effectiveness and meeting 
institutional goals, there remains the equally pressing mandate of shrinking resources and a 
decline in public commitment to higher education. As Nelson has recently noted : "No programs 
are safe when confronted by budget cuts, diminishing state support, higher tuition, threats to 
state and federal student financial aid programs, reduced commitment to provide higher 
education to all citizens, reduced commitment to affirmative action, etc. (Nelson, 'What is 
Happening,' 1995. p. 133)." Such concerns at Appalachian State University resulted in the 
formation of a Program Evaluation Committee [PEC] (on which I have served for three years 
and now chair). In his memorandum of invitation, Provost Harvey Durham charged us as 
follows: 

... It has become increasingly clear to many of us that we cannot continue with 
"business as usual," given the permanent loss of millions of dollars from our 
budget. . . .we must examine our academic programs, both undergraduate and 
graduate, as well as other activities to determine which should be curtailed so that 
those which remain may be adequately supported . We will also need adequate 
funding over the next few years to strengthen some programs and develop some 
new ones (Memorandum, January 27, 1992). 
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Resource reallocation is a direct purpose of the program evaluation procedure instituted 
at Appalachian State University. No department or program enjoys the prospect of undergoing 
such review. The experience, however, has a potential advantage for departments or programs. 

The review process, or similar evaluation programs at other institutions, can offer an ancillary 
opportunity for the assessment of departmental or program effectiveness. This paper examines 
the assessment potential of this program evaluation process. Following an explication of the 
process itself and the PEC Report, five elements of the final report (centrality to university's 
mission, department overview, faculty workload/reassigned time and scholarly activity, program 
demand, and duplication ) are examined for their assessment potential to a department being 
reviewed. My responsibilities are those of Chair of the Department of Communication. 

Program Evaluation Committee Structure 

Appalachian State University's program evaluation process is a faculty designed effort. 

In 1991, evaluation guidelines were developed by the Faculty Senate. A draft was distributed to 
ail department chairs and Faculty Senators, two open faculty forums were held at which the 
draft was considered, and the Faculty Senate debated and passed the final document; the 
"Academic Program Review Evaluation Criteria (see Appendix)." Its primary objective was to 
"... give each academic department the chance to show the strengths as well as the weaknesses 
of its programs and courses. It is an attempt to look more closely at the effectiveness and 
efficiency of each program and to give the department some insight into areas which need 
improvement (Faculty Senate, 1991)." The Program Evaluation Committee, appointed the 
following January, follows these evaluation guidelines. 

Needless to say, the program review process is neither quick nor efficient. It allows the 
eight members of the PEC to consider four or five departments a semester. The PEC begins its 
review process by requesting several documents. The academic department being reviewed is 
invited to provide a written response to the "Academic Program Review Evaluation Criteria" 
document and requested to provide the committee with its last three annual reports, any 
planning documents, any accreditation materials (including material prepared for the most recent 
SACS accreditation) and any other documents the department wishes the committee to consider. 

(Should a chair feel that these other documents are sufficient, she / he may elect not to complete 
the "Academic Program Review Evaluation Criteria" document.) The Office of Institutional 
Research [OIR], in addition to having a member of its staff serving ex officio on PEC, provides 
the committee with a printout, "Faculty and Their Teaching Load," covering the current 
semester and two to three past semesters of a department's teaching load broken down by 
faculty, course, section, hours credit and enrollment. A copy of the "Teaching Load" document 
is also sent to the appropriate department chair. The latest "Workload Data" report, distributed 
to each department annually, is also provided by OIR. This document is a spread sheet showing 
credit hours generated, hours taught by full-time faculty, reassign time for full-time faculty, the 
number of students enrolled in a department, several indices for these statistics and other similar 
statistical material. The report allows the committee to make faculty workload comparisons 
among programs and departments. 

Once the material is assembled, a two-person team from PEC reviews the material 
carefully, consults additional information it may need (such as a job-outlook report), interviews 
the department chairperson for clarifications or additional information and discusses its findings 
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with the whole committee. The team then drafts an initial report following a similar template; 

1. Centrality to Appalachian's Mission, 2. Departmental Overview, 3. Faculty Workload analysis, 
4. Program Demand analysis, 5. Program Duplication information, 6. Faculty Release Time and 
Scholarly Activity analysis, 7. Summary of Findings and 8. Recommendations. This draft is 
presented to the whole committee where it is discussed, amended if necessary and possibly 
redrafted at the committee's request. An approved draft is then sent to the appropriate 
department chair along with any supporting documents. The chair has 30 days to discuss it with 
her/his faculty, meet with the PEC (if so desired), and draft a response. A final report along 
with the department's response is forwarded to the Provost and Vice Chancellor for Academic 
Affairs who forwards it to the appropriate department chair and dean. Finally, a copy of the 
report along with responses from the department and dean are submitted to the university's 
Academic Policies and Procedures Committee for consideration, a possible response, and/or a 
vote of acceptance. Based upon this process, the Provost makes a final decision on the report's 
recommendations and notifies all appropriate persons and groups. 

Elements of the PEC Report 

Centrality to university's mission is the first element of the "Evaluation Criteria" (see 
Appendix) and the initial unit of the PEC Report. Given the reallocation recommendation 
mission of the PEC, centrality becomes a foundation basis for the balance of the report. It 
considers a department's importance to the institution as well as the possibilities of program 
consolidation or program elimination. Of course, all departments believe their programs to be 
important to the university's mission, most can generate arguments attesting to a need of some 
sort, and few departments will admit that programs can be combined or eliminated. Because it is 
difficult to operationalize and might be self-serving, the question of a department's or program's 
centrality to the university is potentially problematic (Nelson, "What is Happening," 1995, p. 

134). Indeed, most responses coming to the committee are inherently self-serving and are 
grounded on any number of idiosyncratic criteria. Herein lies the value of the OIR reports, the 
supporting documents provided by the department (such as annual reports), additional sources 
consulted by committee members, and the interview with the chairperson. 

In most instances, centrality claims can be readily verified. A claim of centrality based 
upon student load, class sizes, or courses required across the curriculum is readily verified by 
the OIR's statistics and other institutional documents. If the statistics do not support a claim, 
annual reports or the conversation with the chair can clarify any possible error or offer an 
alternative explanation. Similarly, the PEC draws upon its independent knowledge of the 
institution or supporting documents (such as the Occupational Outlo ok Handbook ) in 
considering a department's claim regarding the larger question of a discipline's critical 
importance to the institution. The departmental response to the report might further resolve any 
discrepancies or advance explanations. If so, an amended report will be drafted. From the 
varied materials considered in the review process, the PEC is able to engage in a "reality check" 
on a department's centrality claim. However problematic the issue of centrality, departmental 
claims can be resolved. 

Material upon which centrality is based as well as other general information about a 
department or program is reviewed in the Department Overview unit. Primarily descriptive in 
nature, this component of the PEC report is a commentary on a department's degree offerings. 
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courses required for other degree programs across the institution, co-curricular, honor society 
and or professional involvement, and special faculty or student accomplishments. This material, 
while not evaluative per se. presents foundation information upon which other judgments are 
made by the PEC. For example, the centrality judgment can be based, in part, upon the data 
presented in the overview such as the demographics of a program's student cohort or the 
uniqueness of a particular degree. Likewise, data on the number of majors can become a basis 
for understanding program demand. Similarly, other evaluative judgments on faculty workload 
and program duplication as well as the PEC's recommendations are developed from the material 
presented in this unit. Its overview nature gives those reading the final report a general 
understanding of a department or unit and it provides the department with a basis from which to 
understand the findings and recommendations of the PEC Report. 

In light of legislative and public concerns about accountability and responsive to its 
mission regarding resource re-allocation, the Program Evaluation Committee addresses part of 
its report to the broad question of faculty productivity in its unit on Faculty Workload / 
Reassigned Time and Scholarly Activity . Quite simply, the PEC seeks to discover how 
faculty are utilized in a given department and to provide some basis for recommendations 
regarding the need (or lack of need) for additional faculty resources. Necessarily, this inquiry 
involves the instructional workload of a faculty, how and to what extent faculty are reassigned 
from instruction and a faculty's scholarly productivity. 

The initial examination of Faculty Workload begins with OIR statistics and an 
examination of a department's annual reports. The OIR's "Teaching Load" report shows a 
precise student head count in each course offered by the department each semester and the 
number of courses an instructor is teaching. In addition to faculty head count, faculty FTE and 
student credit hour (SCH) statistics, OIR's "Faculty Workload Report" indexes departments on 
enrollment per FTE, SCH per FTE, and average instructional hours per full-time faculty. It also 
lists twelve categories (e g., academic administration, co-curricular program direction, sponsored 
and non-sponsored research, on-campus service, etc.) within which a department can report 
faculty reassignment. (At Appalachian State, a full teaching load is 12 hours per semester.) 
Further details regarding scholarly activity and faculty reassignment are found in the annual 
reports provided by a department. These documents enable the PEC to compare one 
department's workload with other departments in its College and in the University. They also 
provide a foundation for conclusions to be drawn regarding a department's need for additional 
faculty. In its recommendations regarding my department, additional faculty were justified 
because: "The department served 29% of the students enrolled in the CFAA [College of Fine 
and Applied Arts] with 17% of the College's faculty resources in 1992-93 (Program Evaluation 
Committee, 1993, December)." 

While the workload related issues inquire into a department's faculty productivity, the 
Program Demand unit focuses upon a program's viability largely from the perspective of its 
student cohort. To get at program demand, the "Evaluation Criteria" questionnaire invites a 
department to provide demographic information on the number of majors, the number of 
graduates, over or under enrolled classes, information on job prospects for its graduates and how 
its courses are essential to other programs (see Appendix). The PEC augments this information 
with OIR information (contained in the university's Fact Book) on the number of declared majors 
and graduation rate over several semesters as well as information on employment opportunities 
from the Occupational Outlook Handbook. The resulting body of data allows the PEC to draw 
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judgments regarding the future potential of a department's programs and courses as well as how 
well they are serving the student body. 

Program demand also enhances the PEC's understanding of other issues such as a 
program's centrality or faculty workload and, in turn, influences its recommendations. Low 
scholarship productivity or service productivity may be because a department's program demand 
is so heavy that adequate reassigned time is not available. Overcrowded classes or over 
demanded courses and high reassigned time for scholarship or service may invite a 
reexamination of a department's reassigned time policies. Low program demand and a 
relatively low productivity may put program continuation at risk. If this situation is 
accompanied by a judgment of high centrality, a very different analysis could result. Inquiries 
into program demand provide the PEC with a valuable body of information. A more complete 
picture of a department's function within the institution emerges and, hopefully, reallocation 
recommendations are better informed. 

A final specific unit in the PEC Report is Duplication. Like program demand, 
duplication considers a program's or department's utilization of resources but from an efficiency 
perspective. In order to assess duplication related matters, the "Evaluation Criteria" (see 
Appendix) seeks information on how a program compliments or duplicates other programs in the 
institution, whether its programs can be accomplished by other programs and what efforts have 
been made to reduce duplication. This information is augmented by the PEC with the University 
Catalog , providing information on course duplication, and the " Academic Program Inventory 
Summary," a UNC system document providing information on program duplication within the 
system. 

Course duplication is potentially problematic and will be noted in the PEC report. Since 
the institution's course approval process requires a consideration of course duplication, little 
duplication of this nature emerges. Where it is found, the departmental response and the PEC 
Report is likely to be explanatory in nature (i.e., why other courses caimot serve the needs of a 
particular program). Program duplication may not be problematic if the program is otherwise 
thriving. If a program is relatively small but unique within the system and/or valuable to the 
institution's mission, a lack of duplication could be reported as a potential asset. Beyond 
reporting the data and drawing judgments consistent with resource reallocation, the PEC Report 
generally has little to say regarding duplication. 

The final two elements in the PEC Report, the Summary of Findings and 
Recommendations, are of considerable interest to a department being reviewed and the 
administrators to whom the report is submitted. The assessment value of each comes when a 
department is invited to respond -- to agree, disagree, justify or explain. Generally, these 
responses, like the summary and recommendations themselves, are based on material considered 
earlier in the Report. 

The Program Evaluation Process as Assessment 

Conventional wisdom suggests that appropriate assessment should be formative whereas 
the program evaluation process is essentially summative. However, in establishing the 
"Evaluation Criteria," the Faculty Senate did note that one of its purposes was "to give the 
department some insight into areas which need improvement (Faculty Senate, 1991)." In other 
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words, the process itself offers departments a potential opportunity for program assessment. As 
Higgerson suggests, it is an opportunity a department should take: 

Institutions and faculty have a choice. They can respond to external mandates 
and create a bureaucratic system of assessment which is managed separately from 
the academic enterprise or they can invest the extra effort needed to develop an 
effective assessment program which is owned and managed by the faculty and, 
therefore, integrated closely with the academic program (1993, p. 9). 

Although the program assessment opportunity offered by the program evaluation process 
goes well beyond student learning, a traditional focus of program assessment (Higgerson, 1993, 
p. 1), it does reflect assessment potential. The overall process itself has credibility because it 
was developed and is managed by the faculty. Its credibility with departments under review is 
further enhanced because of the opportunity to interact with the PEC and because a standing 
faculty committee (The Academic Policies and Procedures Committee) reviews the PEC Report. 
Additionally, the data reviewed through the process are generally pre-existing data collected or 
generated for other purposes. The widespread institutional involvement with the process as well 
as the convenience of using existing data should encourage departments to use the process as an 
opportunity for assessment. To date, some have done so, resulting in several changes being 
made by departments ranging from deletion of nonproductive programs to alterations in faculty 
workload policies. Finally, improved programs, inter-departmental interaction, and 
departmental enhancement are potential outcomes of effective assessment, all of which benefit 
the institution. Faculty ownership, the perception of assessment as an opportunity, the ability to 
use existing data, a climate of trust, and institutional utility are all components of what 
Higgerson (1993) describes as effective assessment. Additional assessment potential can be 
found in a consideration of specific elements of the PEC Report. 

It is axiomatic that any meaningful assessment of a department's mission, goals or 
accomplishments must be grounded on some contextual placement of the department within its 
institution. Gathering and reviewing the data reflected in the Departmental Overview provides 
an opportunity for understanding this contextual placement. In addition to reviewing the OIR 
data supplied to the PEC, departments are encouraged to respond to a variety of matters such as 
assessment procedures for students and faculty, accreditation review findings, faculty expertise, 
number of majors, number of graduates, and course enrollment. A department confronts, 
therefore, a variety of potentially critical issues. 

Who are the department's constituents? How complete or broadly applicable is its 
curriculum? What assessment methods are used by the department and what are the results? 
What is the role of extra-curricular or co-curricular experiences within the department and how 
do these influence the curriculum and faculty workload? Each of these are critical components 
of the context in which effective program assessment must happen (Galvin, 1992, pp. 22-24). 

A department is not just gathering data (much of which is available from OIR). If it approaches 
the process effectively, it is confronting facts which should educate its own understanding about 
itself Later, as these facts are used in the formation of further information required by the 
evaluation process, the department can gain added insight into such matters as the nature of its 
faculty's workload, its scholarly productivity and the nature of and basis for its program demand. 
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While considering the overview data potentially enables a department to place itself 
within an institutional context, examining its Centrality to University's Mission invites a 
department to stop and consider itself; an introspective review with broader assessment 
potential. In a very real sense, considering how it fits the university mission can be an 
opportunity for the department undergoing review ( Higgerson, 1993, pp. 4-5). "In the best of 
all worlds," Hunt suggests, " there would be no difference between what the institution is trying 
to accomplish and what the department is attempting to do (1992, p. 12)." While the committee 
comes to an educated judgment and evaluation regarding a department's centrality to the mission 
of the university, the same question provides an assessment opportunity for the department 
through the challenge of self examination and analysis. 

Considering its centrality to the institution strongly encourages a department to view 
itself in a comparative manner as implied by Hunt. A department could consider the 
undergraduate mission of the university and how its programs serve undergraduate education. 
Gathering or reviewing data regarding undergraduate enrollment in its courses or how its courses 
serve the larger undergraduate population could inform a department's understanding of 
centrality. Since departmental mission statements are a component part of the armual reports 
reviewed by PEC, this position is made salient to the department. Likewise, a department could 
seek to demonstrate how a particular program's orientation serves a regional cohort among the 
student body or how its program is the only one in the western area of North Carolina. A 
consideration of potential program elimination possibilities invites, if not compels, departments 
to review the value of its programs and to muster evidence in support of that value. For the 
department, the review process invites a dialogue of defense and verification yielding insight 
into the relationship between goals and accomplishments. While not the "best of all worlds," 
this element of the process is a component of effective assessment which must begin with an 
understanding of mission "articulated through . . . goals and objectives (Galvin, 1991, p. 21)." 

The overview and centrality components of the PEC Report give departments the 
opportunity for a broad assessment of its place and role within the larger institution. This 
appraisal, while valuable per se. provides a background from which to understand more specific 
elements relative to departmental or program assessment: Faculty Workload/ Reassigned 
Time and Scholarly Activity, Program Demand, and Program Duplication. As shall be 
shown, these more specific elements not only interact with each other but reveal additional 
insights into departmental effectiveness. 

During the PEC review process, several components of faculty productivity emerge as 
the OIR reports are examined and departmental responses to the "Evaluation Criteria" are 
reviewed. Interrelated with each other, these include Faculty Workload/Reassigned Time and 
Scholarly Activity. Since OIR reports are required to be copied to a department at the initial 
stages of the review, the information available to the PEC is also available to the department. 
Examining this data, in addition to responding to the "Evaluation Criteria" questions and/or a 
draft of the PEC Report, gives a department an additional opportunity to assess itself in terms of 
work load, faculty utilization, scholarship opportunities and productivity. 

Faculty workload is frequently revealed through an examination of statistics regarding 
the number of students in classes, the number of faculty teaching in a program, an average 
faculty teaching load across a department or program and the amount of faculty reassigned time 
and the reasons for the reassignments. All of this data invites a department to confront workload 
issues directly. While faculty may have an individual sense of how much they are working, the 
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perception of such data across the department or program provides potentially functional 
insights. Additional insights emerge when departmentally known data and generated responses 
to that data are compared with the statistical data in the OIR reports. Departments have the 
opportunity to identify and to explore any discrepancies. In light of this appraisal, other 
questions may become salient such as the degree to which a department's faculty are maintaining 
expertise within the field, or whether scholarship productivity is meeting accreditation standards, 
or the overall cost and value of departmental reassigned time. These questions might encourage 
a department to fine tune its reassigned time or class load policies or to think through such 
additional issues as use of reassigned time and/or scholarship opportunities. For example, a 
department with high class loads and low scholarly productivity might recognize that too much 
reassigned time is dedicated to university service. Conversely, low instructional productivity 
may result from a overemphasis on scholarship (a concern confronting many in higher education 
today). 

Whether a department's appraisal occurs as it prepares its initial submission to the PEC 
or reacts to PEC's findings and evaluations, it participates in an internal or external dialogue 
regarding workload related issues. Faculty productivity is highlighted. If a department's 
productivity is found to be or evaluated as high, it is in a position to accept the findings and seek 
to turn them into a request for additional resources. A revelation of low productivity in any of 
the traditional areas of instruction, scholarship or service might lead a department to reassess 
itself and seek justifications or correctives. During the course of its review of departments, the 
PEC process itself has stimulated several departments to develop and implement internal 
correctives measures having an impact on productivity. Whatever a department's response, the 
process invites a broader consideration of a program beyond instruction outcomes per se. a 
productivity evaluation which "ultimately should be a part of program assessment (Denver 
Conference, 1992, p. 11)." 

Traditional program assessment is guided by concerns for a program's curriculum and 
concerns for accountability (Higgerson, 1993, p. 1). The process of the inquiry into Program 
Demand offers additional opportunities for a department in the assessment of both of these 
concerns. As was the situation with workload assessment, the department gathers and reviews 
the data and seeks an understanding behind the statistics and facts. Again, this can happen as it 
prepares for the PEC review or as it responds to the PEC Report. 

The raw data that measures program demand offers potential insight into a number of 
curriculum related matters. For example, classes with consistently low enrollment may evidence 
a course that is offered too many times or in need of updating. Or, it may be a sign of a problem 
with instruction or a course offered at the wrong time of day. If the number of majors is 
declining over time, a department may need to initiate an inquiry into the structure of its major, 
possible recruitment measures or even the continuing need for the major itself On my campus, 
a decline in students majoring in business and a pending PEC review prompted several 
departments in the college to revise curriculum, to step-up recruitment and to open some courses 
to non-majors. Conversely, departments faced with a high student demand might wish to 
investigate some means of limiting majors, minors or service students. The popularity of the 
communication/mass communication major has led my department to take this step in the past 
and is leading many others to take similar measures. Similarly, a decline in the number of 
graduates (usually accompanying a decline in the number of majors) is an additional signal for a 
department to initiate a careful reassessment of its curriculum. Each of these and other potential 
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instances denote an opportunity if not a responsibility for a department to further assess what it 
is doing in terms of its curriculum, its student cohort and its institution. 

Is the funding dollar wisely spent? Are graduates prepared and able to get jobs? Does a 
program have enough majors to justify its continuation? These and similar questions are basic 
to the issue of accountability (Higgerson, 1993, p. 2). As the PEC review inquires into program 
demand, the process provides ample opportunity for departmental appraisal. For example, it 
invites a department to examine (indeed defend) its graduates' job prospects. Each PEC Report 
seeks data on job prospects from the Occupational Outlook Handbook (if available). A 
department would wisely make such an inquiry itself in preparation for review or in response to 
the Report. Low class enrollment, few majors, and /or few graduates raise significant 
accountability questions as do crowded or over demanded courses. On my campus, several 
programs were voluntarily terminated following an appraisal of low demand or output. Program 
demand along with faculty productivity also inform the analysis of a department's centrality. For 
a variety of fundamental reasons, how well a department is using its resources or serving its 
majors is an essential assessment question. 

A final component of the PEC Report with assessment potential for a department is the 
inquiry into Program Duplication. As a department responds to the four questions posed in the 
"Evaluation Criteria" or prepares its response to a draft of the PEC Report, it has an opportunity 
to examine its program or programs in comparison with and contrast to other units in the 
institution or throughout the system of which it is a part. If courses are shown to be duplicated, a 
variety of departmental responses exist. Duplication in the face of a high workload or low 
student demand might lead a department to consider eliminating its own course(s). However, a 
department's course that appears to duplicate in another department may be justified on the 
grounds of its uniqueness to a department. At my institution, a number of departments offer 
research methodology courses, each uniquely tailored to the particular needs of the respective 
department. 

The revelation of programs with common objectives could be a stimulus for a 
department either to review its own programs' objectives or to seek out interactive relationships 
with other units. To discover and report these interactions, enhances the perception of a 
department's strength and may educate future departmental curriculum decisions. Unless a 
department offers a core course in a general studies curriculum, it may not know which of its 
courses support programs in other departments. Program interaction, an asset on most campuses 
(Nelson, "What is Happening," 1995, p. 134; Bettinghaus, 1995, p. 137), might be a useful 
discovery to come from this process. Confronting program duplication or uniqueness within a 
system might produce insights such as a department's particular niche among competing 
programs or why its uniqueness is of value to its institutional mission. 

The discovery of program or course duplication need not be the "kiss of death" for a 
course or program. While institutional or departmental managers might seek to reduce 
duplication as a way of improving efficiency, the revelation of (and possible justification for) 
duplication could ultimately prove productive to a department or program. Again, the process 
provides an opportunity for departmental inquiry into valuable relationships which go well 
beyond the immediate resource allocation objective of the PEC. Whatever the outcome, it 
contributes to a department's larger assessment of itself and how it relates to its institution. 
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The Program Evaluation Committee was not established as an assessment agency per se . 
Its direct mission is summative rather than formative. The institution wished to have some basis 
for making internal resource reallocation decisions and the PEC Report on each program informs 
those judgments. The process, however, need not be feared by departments under review at 
Appalachian State University or whatever other campus such evaluation is happening, it should 
be seen as an opportunity. The same motivation which drives an institution to evaluate its 
programs and departments should encourage the same units to appraise themselves. This paper 
illustrates how this or similar program evaluation processes have the potential for assessing 
departmental effectiveness for departments wishing to take advantage of the opportunity. 
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Appendix : Academic Program Review Evaluation Criteria 



1. CENTRALITY TO UNIVERSITY'S MISSION 

How important to the mission of Appalachian State University is this program as a 
undergraduate/graduate degree program? 

Is there a cultural/societal need for the program? 

Can this program be combined with a similar pro^am in another department? 

Can this program be eliminated by the department? 

2. PROGRAM 

What is the quality of the program as perceived by the students? Please indicate how this 
perception of the program is obtained. 

How effective is the mechanism used to assess the program's quality and direction? 

If the program is accredited or has applied for accreditation attach the summary report. 

Is there a mechanism beyond the university's admission requirements for admission to the 
program? If so, please describe it. 

If such a mechanism exists, how effective is it in assessing the students entering the 
program? 

Is there a mechanism in place to assess the students graduating from the program? If so, 
please describe it. 

How effective is the mechanism? 

In what way does the number of students in a given course/section affect the learning 
experience of the student? 

Is there a locational (geographic/demographic/cultural) advantage to the program? 

Does the program contain any unique courses? If so, please elaborate. 

What indications do you use to assess the quality of the program as perceived by 
significant publics (e.g. high school counselors, community college advisors, employers, 
potential graduate students, industry recruiters)? 

3. FACULTY 

How many faculty members are teaching in this program? 

How many graduate assistants and part-time instructors are usually teaching in this 
program? 

How many of the faculty members have appropriate terminal degrees? 

How many of the faculty members who do not have appropriate terminal degrees have 
attained equivalencies in their own or related fields? 

Is instruction assessed? If so, how is it assessed and what are the results? 

Does the faculty maintain/develop expertise in areas appropriate to their field/specialty? 
(If this is clearly indicated in your department's annual report you need not respond here.) 
What is the average teaching load of the faculty in the department? 
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What is the average percentage per semester of reassigned faculty time? 

If this program should be discontinued, can the faculty members teaching in it be placed 
in a related area? If the answer is yes, please indicate the area. 

4. LIBRARY HOLDINGS 

Does the library have the resources, current and retrospective, to support the program? 

At what level(s); undergraduate, graduate, research? 

Is there accessibility to appropriate information at other locations? 

5. FACILITIES/EQIJIPMENT 

Is the available space (classroom, laboratory, etc.) adequate for the program? At what 
level(s): undergraduate, graduate, research. 

How appropriate is the space? At what level(s); undergraduate, graduate, research? List 
changes/improvements necessary to improve rating and indicate for which level(s). 

Is the necessary equipment available? Indicate for which level(s): undergraduate, 
graduate research? 

Is the available equipment in working order? Indicate for which level(s): undergraduate, 
graduate, research? 

6. DEMAND 

Is the program less than 5 years old? If it is, please indicate the year it came on line and 
the predicted numbers of students that this program is intended to serve? 

How many majors/graduate students are in the program? 

How many graduates does the program produce? 

What are the job prospects for these graduates? (Based on occupational outlook or other 
appropriate data.) 

Are courses in the program essential to other programs? 

Are there courses that are consistently over enrolled? If so, please list the courses. 

Are there courses that are under enrolled? If so, please list the courses. 



On the average, do undergraduate students in the program graduate with more than the 
university required number of hours. If so, what is the average number of hours program 
graduates have on graduation? 

Of the above total, how many are required hours for the program? 

If the program leads to a degree, how many concentrations are available? 

How effectively does the program meet accreditation requirements? (Based on # of 
required hours in major as compared with the # required at other institutions in the UNO 
system and or the # required hours at similar institutions outside of the UNC system.) 

Has the department considered/instituted a requirement for a minimum number of 
students to be registered before a course will be offered? If yes, what is that minimum? 
Given the threshold of less than 10 for undergraduate courses and less than 5 for graduate 
courses, does the program have under enrolled courses? If so, please list the courses and 
indicate what steps have been taken to reduce the frequency of offering those courses? 
Briefly list other cost saving measures and the results of each. 

Does the program require specialized equipment? Is this specialized equipment 
necessary? Estimate the yearly costs of the equipment, including maintenance. 

If current equipment/facilities are not adequate, what would it cost to make them 
adequate? Superior? 



7. COSTS 
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What are the costs of library acquisitions for support of the program? 

If library resources are not adequate, what would it cost to make them adequate? 

Superior? 

What is the cost of faculty reassigned time, including the administration of special 
programs/institutes? (Include the costs of overhead, such as special equipment, space, 
etc.) 

Would the department rather spend the above dollars on other endeavors? Please 
indicate why. 

8. DUPLICATION 

Does the program complement others at the university? Briefly indicate how. 

Can this program's objectives be accomplished equally well through another program? 
Briefly explain. 

Are courses in the program duplicated in other programs/departments? If yes, please 
indicate other programs/departments. 

Has there been an effort to eliminate the duplication of these courses? If yes, briefly 
indicate the extent of the effort. 

Is the program distinctive in the UNC system? If yes, briefly indicate why/how. 

9. CRITICAL MASS 

What is the impact on the department if the program is eliminated? 

Would the department still exist? For what purpose(s)? Briefly state other purposes. 

What is the impact on other department/programs? 

Would a change in/elimination of the program affect the ability of the college/department 
to carry out its mission? 
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